The general problem of computerized object detection in images has been studied for decades, and many techniques are known in the art. Although the entire field of object recognition (“OR”) has experienced some recent advances, OR algorithms have historically suffered less than adequate performance in correctly identifying objects in visual imagery streams, especially with respect to radar imagery. In addition, OR algorithms have historically required significant computational resources, such as high performance computing (“HPC”) clusters, which have significant space and power requirements, as compared to small size, weight, and power (“SWAP”) technologies.
In part because of their space and power requirements, whenever OR algorithms have been applied to imagery obtained from devices installed in airborne vehicles (including space-borne vehicles), the OR algorithms have been hosted largely on ground stations. In these applications, images gathered by airborne vehicles have been transmitted to the ground stations via a high-speed wireless downlink. On the ground stations, OR algorithms running on HPC clusters have been employed to identify specific objects in the gathered imagery. Once identified, information describing the identified objects has been communicated from the ground stations back to the airborne vehicles for use in determining specific actions to take with respect to the identified objects. Such architectures have obvious limitations, including: (1) a distributed wireless communications architecture with inherent delays and reliability problems; (2) a ground station with logistics, security, and cost concerns; (3) expensive HPC technology; and (4) expensive manpower required to operate the ground-based devices.
Although different object recognition algorithms can operate on information obtained from different kinds of sensors, most successes in the field of object recognition have been achieved from algorithms that accept data from sensors that employ passive electro-optical (“EO”) technology. Passive electro-optical sensors are relatively common and most can be fielded on very small handheld devices. They are typically a passive imaging modality that can be found in photo and video cameras, motion sensors, and similar devices that convert incoherent light into electronic signals.
Object recognition algorithms that operate on data obtained from electro-optical sensors have traditionally employed a variety of techniques to recognize objects. These techniques are well known in the art and include: edge detection, corner detection, gray-scale matching, gradient matching, pose clustering, feature detection, feature comparison, scale-invariant feature transforms, domain-specific image descriptors and many others.
In contrast to passive, electro-optical (“EO”) image sensing technologies that convert incoherent light into electronic signals, synthetic aperture radar (“SAR”) is (for example, in spotlight mode SAR imaging) a coherent, side-looking radar system that utilizes the flight path of a flying platform to electronically simulate an extremely large antenna or aperture, and thereby obtain high-resolution imagery by illuminating objects with interrogating radar signals. The imagery is generated from a series of radio wave pulses that are transmitted to “illuminate” a target scene, and the echo of each pulse is received and recorded by sensors. The pulses are transmitted and the echoes received (typically) by a single beam-forming antenna, with typical wavelengths of one meter to millimeters. Multiple transmitters and receivers (as in multistatic radar) are also possible. As a SAR device moves on a flying vehicle, the antenna locations relative to a target change over time. Signal processing of the recorded radar echoes allows a SAR device to combine recordings from the multiple antenna locations. The multiple antenna locations together form a synthetic antenna aperture, which allows the device to create a finer resolution image than would be possible with a given physical antenna aperture.
Although the noise statistics of SAR images have historically prevented OR algorithms on SAR imagery from equaling the performance of OR algorithms on EO imagery, SAR imagery has several advantages over electro-optical imagery, especially in the context of object detection and identification, and especially for flying platforms. SAR imaging platforms can visualize objects through clouds, operate at night, at greater distances, and can perform better in certain applications, such as timber assessment, law enforcement, and air traffic monitoring.
However, efforts to apply object recognition (OR) algorithms to SAR imagery have historically been relatively unsuccessful. Algorithms and techniques that have worked well for passive, incoherent EO imagery have not proved as effective for active, coherent SAR imagery. For example, researchers developing object recognition algorithms for SAR imaging data have not been able to successfully incorporate feature detectors like Harris corner detectors and Plessy corner detectors, or feature descriptors like scale-invariant feature transforms (“SIFT”) and Speeded Up Robust Features (“SURF”).
Researchers have also been relatively unsuccessful at utilizing deep learning techniques to identify objects within data obtained from active and coherent radiation devices, such as synthetic aperture radar. Thus, there has been a significant performance gap in object recognition algorithms that operate on passive, incoherent, electro-optical (“EO”) imagery versus active, coherent, SAR imagery, with OR performance on EO imagery typically outperforming OR performance on SAR imagery by a wide margin, even using the same OR algorithms. The OR performance compromise is so great and teaches away from OR on SAR imagery to such a degree that historically, alternative sensors to SAR (like EO sensors) are typically considered more appropriate for tasks that would otherwise be a good fit for SAR imaging, due to the inapplicability of most OR methods to SAR imaging.
Owing to this performance gap, instead of attempting to address deficiencies in object recognition performance on SAR sensor imagery, the conventional approach has been to use an EO imaging sensor (or other sensor or combination of sensors) and incorporate traditional EO object recognition techniques in OR (i.e., target identification) applications. That is, it was considered more economically advantageous to “fix the sensor” (by substituting EO sensors where SAR sensors might be more appropriate) than to fix the OR algorithms operating on SAR imagery. Deployed systems have still exploited SAR imaging for related tasks, for which OR algorithms do provide some benefit, but have used EO technology for more difficult object recognition tasks. Such architectures were adopted despite the fact that EO sensors need to observe objects from closer range to achieve the requisite accuracy.